Oracle inequalities for cross-validation type procedures
نویسندگان
چکیده
Abstract We prove oracle inequalities for three different type of adaptation procedures inspired by cross-validation and aggregation. These procedures are then applied to the construction of Lasso estimators and aggregation with exponential weights with data-driven regularization and temperature parameters, respectively. We also prove oracle inequalities for the crossvalidation procedure itself under some convexity assumptions.
منابع مشابه
Slope heuristics and V-Fold model selection in heteroscedastic regression using strongly localized bases
We investigate the optimality for model selection of the so-called slope heuristics, V -fold cross-validation and V -fold penalization in a heteroscedatic with random design regression context. We consider a new class of linear models that we call strongly localized bases and that generalize histograms, piecewise polynomials and compactly supported wavelets. We derive sharp oracle inequalities ...
متن کاملDiscussion of “2004 Ims Medallion Lecture: Local Rademacher Complexities and Oracle Inequalities in Risk Minimization” by v. Koltchinskii
1. Introduction. This paper unifies and extends important theoretical results on empirical risk minimization and model selection. It makes extensive and efficient use of new probability inequalities for the amount of concentration of the (possibly symmetrized) empirical process around its mean. The results are very subtle and very pleasing indeed, as they show that oracle inequalities exist for...
متن کاملChoosing a penalty for model selection in heteroscedastic regression
Penalization is a classical approach to model selection. In short, penalization chooses the model minimizing the sum of the empirical risk (how well the model fits data) and of some measure of complexity of the model (called penalty); see FPE [1], AIC [2], Mallows’ Cp or CL [22]. A huge amount of literature exists about penalties proportional to the dimension of the model in regression, showing...
متن کاملDensity estimation via cross-validation: Model selection point of view
The problem of model selection by cross-validation is addressed in the density estimation framework. Extensively used in practice, cross-validation (CV) remains poorly understood, especially in the non-asymptotic setting which is the main concern of this work. A recurrent problem with CV is the computation time it involves. This drawback is overcome here thanks to closed-form expressions for th...
متن کاملOptimal regression rates for SVMs using Gaussian kernels
Support vector machines (SVMs) using Gaussian kernels are one of the standard and state-of-the-art learning algorithms. In this work, we establish new oracle inequalities for such SVMs when applied to either least squares or conditional quantile regression. With the help of these oracle inequalities we then derive learning rates that are (essentially) minmax optimal under standard smoothness as...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2011